ChatGPT can now see, hear, and speak: A new update adds image and voice capabilities to the AI chatbot

ChatGPT, the popular AI chatbot, has been upgraded with new image and voice capabilities. This means that ChatGPT can now view and analyze images, and also provide voice output. This brings ChatGPT closer to AI assistants we've seen in sci-fi movies.

OpenAI, the company behind ChatGPT, announced the new features on Monday, September 25, 2023. They said that the new voice and image capabilities in ChatGPT offer a "more intuitive type of interface by allowing you to have a voice conversation or show ChatGPT what you're talking about.

The new features are available to Plus and Enterprise users of ChatGPT, and will be rolled out over the next two weeks. Voice is coming on iOS and Android (opt-in in your settings) and images will be available on all platforms.

How to use the new voice feature

To use the new voice feature, you need to go to Settings → New Features on the mobile app and opt into voice conversations. Then, you can tap the headphone button located in the top-right corner of the home screen and choose your preferred voice out of five different voices.

The new voice feature allows you to use your voice to engage in a back-and-forth conversation with ChatGPT. You can speak with it on the go, request a bedtime story, or settle a dinner table debate. ChatGPT will respond to you with a human-like voice, powered by a new text-to-speech model that can generate audio from just text and a few seconds of sample speech. OpenAI collaborated with professional voice actors to create each of the voices. They also use Whisper, their open-source speech recognition system, to transcribe your spoken words into text.

To use the new image feature, you can either snap a picture or choose an image from your gallery. On your iOS and Android phones, you need to tap the plus button first. You can also add multiple images by tapping the plus button again or using the drawing tool to guide your assistant.

The new image feature allows you to show ChatGPT one or more images and have a conversation about them. ChatGPT can apply its language reasoning skills to a wide range of images, such as screenshots, photographs, and documents, with both text and images. You can use this feature to troubleshoot why your grill won't start, explore the contents of your fridge to plan a meal, or analyze a complex graph for work-related data. You can also focus on a specific part of the image by using the drawing tool in the mobile app.

Why the new features are important

The new features are important because they make ChatGPT more versatile and accessible. By adding voice and image capabilities, ChatGPT can interact with users in more natural and intuitive ways, and also handle more complex and diverse tasks. This can enhance the user experience and satisfaction, and also open up new possibilities for using ChatGPT in various domains and scenarios.

The new features are also important because they demonstrate the progress and potential of AI technology. ChatGPT is one of the most advanced AI chatbots in the world, powered by a large-scale neural network that can generate coherent and engaging text from any prompt. By adding voice and image capabilities, ChatGPT shows that it can also learn from and leverage other modalities of information, and integrate them with its language skills. This can make ChatGPT more intelligent and adaptable, and also inspire new research and innovation in the field of AI.